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Mr. Chairman and Members of the Subcommittee: 



I am pleased to be here today as you examine issues related to girls’ 
delinquency — a topic that has attracted the attention of federal, state, and 
local policymakers for more than a decade as girls have increasingly 
become involved in the juvenile justice system. For example, from 1995 
through 2005, delinquency caseloads for girls in juvenile justice courts 
nationwide increased 15 percent while boys’ caseloads decreased by 12 
percent. 1 More recently, in 2007, 29 percent of juvenile arrests — about 
641,000 arrests — involved girls, who accounted for 17 percent of juvenile 
violent crime arrests and 35 percent of juvenile property crime arrests. 2 
Further, research on girls has highlighted that delinquent girls have higher 
rates of mental health problems than delinquent boys, receive fewer 
special services, and are more likely to abandon treatment programs. 3 

The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is the 
Department of Justice (DOJ) office charged with providing national 
leadership, coordination, and resources to prevent and respond to juvenile 
delinquency and victimization. OJJDP supports states and communities in 
their efforts to develop and implement effective programs to, among other 
things, prevent delinquency and intervene after a juvenile has offended. 
For example, from fiscal years 2007 through 2009, Congress provided 
OJJDP almost $1.1 billion to use for grants to states, localities, and 
organizations for a variety of juvenile justice programs, including 
programs for girls. Also, in support of this mission, the office funds 
research and program evaluations related to a variety of juvenile justice 
issues. 

As programs have been developed at the state and local levels in recent 
years that specifically target preventing girls’ delinquency or intervening 
after girls have become involved in the juvenile justice system, it is 
important that agencies providing grants and practitioners operating the 



3 C. Puzzanchera and W. Kang, Juvenile Court Statistics Databook (2008), 
http://ojjdp.ncjrs.gov/ojstatbb/jcsdb/ (accessed Oct. 15, 2009). This Web site provides the 
most current data available. 

2 C. Puzzanchera, Juvenile Arrests 2007, (2009) www.ncjrs.gov/pdffilesl/ojjdp/225344.pdf 
(accessed Oct. 15, 2009). 

3 Elizabeth Cauffman and others, “Gender Differences in Mental Health Symptoms among 
Delinquent and Community Youth,” Youth Violence and Juvenile Justice, vol. 5, no. 3 
(2007): 287-307. Elizabeth Caufmann “Understanding the Female Offender” The Future of 
Children, vol. 18, no. 2 (2008): 119-142. 



Page 1 



GAO-10-133T 




programs have information about which of these programs are effective. 

In this way, agencies can help to ensure that limited federal, state, and 
local funds are well spent. In general, effectiveness is determined through 
program evaluations, which are systematic studies conducted to assess 
how well a program is working — that is, whether a program produced its 
intended effects. To help ensure that grant funds are being used 
effectively, you asked us to review OJJDP’s efforts related to studying and 
promoting effective girls’ delinquency programs. We issued a report on the 
results of that review on July 24, 2009. 4 My statement today, as requested, 
highlights findings from that report and addresses (1) efforts OJJDP has 
made to assess the effectiveness of girls’ delinquency programs, (2) the 
extent to which these efforts are consistent with generally accepted social 
science standards and federal standards to communicate with 
stakeholders, and (3) the findings from OJJDP’s efforts and how the office 
plans to address the findings. 

My statement is based on our July report and selected updates made in 
October 2009. 5 For our report, we reviewed documentation about OJJDP’s 
establishment of a study group to assess the effectiveness of girl’s 
delinquency programs, analyzed the groups’ activities and findings, and 
interviewed OJJDP research and program officials and the current and 
former principal investigators of the study group. Specifically, we 
reviewed the criteria the study group used to assess studies of girls’ 
delinquency programs and whether the group’s application of those 
criteria was consistent with generally accepted social science standards 
for evaluation research. 6 We also compared OJJDP’s efforts with criteria in 
Standards for Internal Control in the Federal Government, specifically 
that agency management should ensure that there are adequate means of 



4 GAO, Juvenile Justice: Technical Assistance and Better Defined Evaluation Plans Will 
Help to Improve Girls’ Delinquency Programs, GAO-09-721R (Washington, D.C.: July 24, 
2009). 

'In a September 18, 2009, letter regarding the recommendation we made in our July report, 
DOJ clarified actions it was taking to address our recommendation, which we have 
included in this statement. 

6 For social science standards for evaluation research, see Donald T. Campbell and Julian 
Stanley, Experimental and Quasi-Experi.mental Designs for Research (Chicago: Rand 
McNally, 1963); William R. Shadish, Thomas D. Cook, and Donald T. Campbell, 
Experimental and Quasi-Experimental Designs for Generalized Causal Inference 
(Boston: Houghton Mifflin, 2002); Carol H. Weiss, Evaluation: Methods for Studying 
Programs and Policies, Second Edition (Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1998); 
and GAO, Designing Evaluations, GAO/PEMD-10.1.4 (Washington, D.C.: May 1991). 
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obtaining information from and communicating with external stakeholders 
who may have a significant impact on the agency achieving its goals, such 
as practitioners operating programs or researchers assessing programs. 7 In 
addition, we conducted interviews with 18 girls’ delinquency subject 
matter experts, that is, researchers and practitioners, whom we selected 
on the basis of their knowledge and experience with girls’ delinquency 
issues. 8 While their comments cannot be generalized to all girls’ 
delinquency experts, we nonetheless believe that their views gave us 
useful insights on issues related to girls’ delinquency and OJJDP’s efforts 
to assess girls’ programs. Our work was performed in accordance with 
generally accepted government auditing standards. More detail about our 
scope and methodology is included in our July report. 9 



OJJDP Established 
the Girls Study Group 
to Assess the 
Effectiveness of Girls’ 
Delinquency 
Programs 



With an overall goal of developing research that communities need to 
make sound decisions about how best to prevent and reduce girls’ 
delinquency, OJJDP established the Girls Study Group (Study Group) in 
2004 under a $2.6 million multiyear cooperative agreement with a research 
institute. 10 OJJDP’s objectives for the group, among others, included 
identifying effective or promising programs, program elements, and 
implementation principles (i.e., guidelines for developing programs). 
Objectives also included developing program models to help inform 
communities of what works in preventing or reducing girls’ delinquency, 
identifying gaps in girls’ delinquency research and developing 
recommendations for future research, and disseminating findings to the 
girls’ delinquency field about effective or promising programs. To meet 



7 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-OO-21.3.1 
(Washington, D.C.: November 1999). 

s GAO defines an expert as a person who is recognized by others who work in the same 
subject matter area as having knowledge that is greater in scope or depth than that of most 
people working in that area. The expert’s knowledge can come from education, experience, 
or both. We identified researchers who focus on girls’ delinquency issues and practitioners 
who operate programs that address girls’ delinquency. Specifically, these 18 experts 
included 11 of the 15 study group members and 7 experts who were not members of the 
group. While we contacted all 15 of the study group members, 4 members either did not 
respond to requests for interviews or declined to be interviewed. 

9 GAO-09-721R. 

10 Cooperative agreements, rather than grant awards, can be used by federal agencies when 
substantial involvement is expected between the agency and the recipient when carrying 
out the activities described in the program announcement. OJJDP extended the 
cooperative agreement with the research institute through June 2010 to complete all of the 
Study Group activities. 
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OJJDP’s objectives, among other activities, the Study Group identified 
studies of delinquency programs that specifically targeted girls by 
reviewing over 1,000 documents in relevant research areas. These included 
criminological and feminist explanations for girls’ delinquency, patterns of 
delinquency, and the justice system’s response to girls’ delinquency. As a 
result, the group identified 61 programs that specifically targeted 
preventing or responding to girls’ delinquency. Then, the group assessed 
the methodological quality of the studies of the programs that had been 
evaluated using a set of criteria developed by DOJ’s Office of Justice 
Programs (OJP) called What Works to determine whether the studies 
provided credible evidence that the programs were effective at preventing 
or responding to girls’ delinquency. 11 The results of the group’s assessment 
are discussed in the following sections. 



OJJDP Efforts to 
Assess Program 
Effectiveness Were 
Consistent with Social 
Science Practices and 
Standards, and OJJDP 
Has Taken Action to 
Enhance 

Communication about 
the Study Group with 
External Stakeholders 



OJJDP’s effort to assess girls’ delinquency programs through the use of a 
study group and the group’s methods for assessing studies were consistent 
with generally accepted social science research practices and standards. 

In addition, OJJDP’s efforts to involve practitioners in Study Group 
activities and disseminate findings were also consistent with the internal 
control standard to communicate with external stakeholders, such as 
practitioners operating programs. 12 

According to OJJDP research and program officials, they formed the Study 
Group rather than funding individual studies of programs because study 
groups provide a cost-effective method of gaining an overview of the 
available research in an issue area. As part of its work, the group 
collected, reviewed, and analyzed the methodological quality of research 
on girls’ delinquency programs. The use of such a group, including its 
review, is an acceptable approach for systematically identifying and 
reviewing research conducted in a field of study. This review helped 



u The What Works criteria define six levels of effectiveness, including effective, promising, 
and ineffective, for use in assessing and classifying studies on the basis of their evidence of 
effectiveness. The criteria for an effective program include a randomized controlled 
research design — a design that compares the outcomes for individuals who are randomly 
assigned to either the program being studied or to a nonparticipating control group before 
the intervention. While other research designs can produce valid results, we have 
previously reported that when it is feasible and ethical to do so, randomized controlled 
designs provide researchers with the best method for assessing a program’s effectiveness 
because they isolate changes caused by the program from other factors. 

12 GAO/AIMD-00-2 1.3. 1. 
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consolidate the research and provide information to OJJDP for 
determining evaluation priorities. Further, we reviewed the criteria the 
group used to assess the studies and found that they adhere to generally 
accepted social science standards for evaluation research. We also 
generally concurred with the group’s assessments of the programs based 
on these criteria. According to the group’s former principal investigator, 
the Study Group decided to use OJP’s What Works criteria to ensure that 
its assessment of program effectiveness would be based on highly rigorous 
evaluation standards, thus eliminating the potential that a program that 
may do harm would be endorsed by the group. However, 8 of the 18 
experts we interviewed said that the criteria created an unrealistically high 
standard, which caused the group to overlook potentially promising 
programs. OJJDP officials stated that despite such concerns, they 
approved the group’s use of the criteria because of the methodological 
rigor of the framework and their goal for the group to identify effective 
programs. 

In accordance with the internal control standard to communicate with 
external stakeholders, OJJDP sought to ensure a range of stakeholder 
perspectives related to girls’ delinquency by requiring that Study Group 
members possess knowledge and experience with girls’ delinquency and 
demonstrate expertise in relevant social science disciplines. The initial 
Study Group, which was convened by the research institute and approved 
by OJJDP, included 12 academic researchers and 1 practitioner; someone 
with experience implementing girls’ delinquency programs. However, 11 of 
the 18 experts we interviewed stated that this composition was 
imbalanced in favor of academic researchers. In addition, 6 of the 11 said 
that the composition led the group to focus its efforts on researching 
theories of girls’ delinquency rather than gathering and disseminating 
actionable information for practitioners. 13 According to OJJDP research 
and program officials, they acted to address this issue by adding a second 
practitioner as a member and involving two other practitioners in study 
group activities. OJJDP officials stated that they plan to more fully involve 
practitioners from the beginning when they organize study groups in the 
future and to include practitioners in the remaining activities of the Study 
Group, such as presenting successful girls’ delinquency program practices 
at a national conference. Also, in accordance with the internal control 
standard, OJJDP and the Study Group have disseminated findings to the 



13 The other seven experts did not express views regarding the balance of the study group’s 
composition. 
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research community, practitioners in the girls’ delinquency field, and the 
public through conference presentations, Web site postings, and published 
bulletins. The group plans to issue a final report on all of its activities by 
spring 2010. 



The Study Group 
Found No Evidence of 
Effective Girls’ 
Delinquency 
Programs; in 
Response OJJDP 
Plans to Assist 
Programs in 
Preparing for 
Evaluations but Could 
Strengthen Its Plans 
for Supporting Such 
Evaluations 



The Study Group found that few girls’ delinquency programs had been 
studied and that the available studies lacked conclusive evidence of 
effective programs; as a result, OJJDP plans to provide technical 
assistance to help programs be better prepared for evaluations of their 
effectiveness. However, OJJDP could better address its girls’ delinquency 
goals by more fully developing plans for supporting such evaluations. 

In its review, the Study Group found that the majority of the girls’ 
delinquency programs it identified — 44 of the 61 — had not been studied by 
researchers. For the 17 programs that had been studied, the Study Group 
reported that none of the studies provided conclusive evidence with which 
to determine whether the programs were effective at preventing or 
reducing girls’ delinquency. For example, according to the Study Group, 
the studies provided insufficient evidence of the effectiveness of 11 of the 
17 programs because, for instance, the studies involved research designs 
that could not demonstrate whether any positive outcomes, such as 
reduced delinquency, were due to program participation rather than other 
factors. Based on the results of this review, the Study Group reported that 
among other things, there is a need for additional, methodologically 
rigorous evaluations of girls’ delinquency programs; training and technical 
assistance to help programs prepare for evaluations; and funding to 
support girls’ delinquency programs found to be promising. 



According to OJJDP officials, in response to the Study Group’s finding 
about the need to better prepare programs for evaluation, the office plans 
to work with the group and use the remaining funding from the effort — 
approximately $300,000 — to provide a technical assistance workshop by 
the end of October 2009. The workshop is intended to help approximately 
10 girls’ delinquency programs prepare for evaluation by providing 
information about how evaluations are designed and conducted and how 
to collect data that will be useful for program evaluators in assessing 
outcomes, among other things. In addition, OJJDP officials stated that as a 
result of the Study Group’s findings, along with feedback they received 
from members of the girls’ delinquency field, OJJDP plans to issue a 
solicitation in fiscal year 2010 for funding to support evaluations of girls’ 
delinquency programs. 
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OJJDP has also reported that the Study Group’s findings are to provide a 
foundation for moving ahead on a comprehensive program related to girls’ 
delinquency. However, OJJDP has not developed a plan that is 
documented, is shared with key stakeholders, and includes specific 
funding requirements and commitments and time frames for meeting its 
girls’ delinquency goals. Standard practices for program and project 
management state that specific desired outcomes or results should be 
conceptualized, defined, and documented in the planning process as part 
of a road map, along with the appropriate projects needed to achieve those 
results, supporting resources, and milestones. 14 In addition, government 
internal control standards call for policies and procedures that establish 
adequate communication with stakeholders as essential for achieving 
desired program goals. 15 According to OJJDP officials, they have not 
developed a plan for meeting their girls’ delinquency goals because the 
office is in transition and is in the process of developing a plan for its 
juvenile justice programs, 16 but the office is taking steps to address its 
girls’ delinquency goals, for example, through the technical assistance 
workshop. Developing a plan for girls’ delinquency would help OJJDP to 
demonstrate leadership to the girls’ delinquency field by clearly 
articulating the actions it intends to take to meet its goals and would also 
help the office to ensure that the goals are met. 

In our July report, we recommended that to help ensure that OJJDP meets 
its goals to identify effective or promising girls’ delinquency programs and 
supports the development of program models, the Administrator of OJJDP 
develop and document a plan that (1) articulates how the office intends to 
respond to the findings of the Study Group, (2) includes time frames and 
specific funding requirements and commitments, and (3) is shared with 
key stakeholders. OJP agreed with our recommendation and outlined 
efforts that OJJDP plans to undertake in response to these findings. For 



14 Project Management Institute, The Standard for Program Management. 

1B GAO/AIMD-00-2 1.3. 1. 

16 OJJDP is required under the Juvenile Justice and Delinquency Prevention Act to publish 
an annual program plan that will, among other things, lay out goals and criteria for 
conducting research and evaluation for its juvenile justice programs. 42 U.S.C. § 5614(b)(5). 
This plan is required to be published annually in the Federal Register for public comment, 
and is to describe the activities the Administrator intends to carry out under Parts D and E. 
Under Part D, OJJDP is authorized to conduct research, evaluation, and technical 
assistance, among other things. 42 U.S.C. §§ 5661-62. Under Part E, OJJDP is authorized to 
make grants for developing, testing and demonstrating promising new initiatives and 
programs. 42 U.S.C. §§ 5665-66. 
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example, OJJDP stated that it anticipates publishing its proposed juvenile 
justice program plan, which is to include how it plans to address girls’ 
delinquency issues, in the Federal Register to solicit public feedback and 
comments, which will enable the office to publish a final plan in the 
Federal Register by the end of the year (December 31, 2009). 

Mr. Chairman, this concludes my statement. I would be pleased to respond 
to any questions that you or other Members of the Subcommittee may 
have. 
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